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4.  LIST  OF  ATTENDEES 


1.  INTRODUCTION 


This  workshop  on  "Quantum-Limited  Imaging  and  Image  Processing" 
represents  the  eighth  of  a  series  of  intensive  academic/  government 
interactions  in  the  field  of  advanced  electro-optics,  as  part  of  the  Army 
sponsored  University  Research  Initiative.  By  documenting  the  associated 
technology  status  and  dialogue  it  is  hoped  that  this  baseline  will  serve  all 
interested  parties  towards  providing  a  solution  to  high  priority  Army 
requirements.  Responsible  for  program  and  program  execution  are 
Dr.  Nicholas  George,  University  of  Rochester  (ARO-URI)  and  Dr.  Rudy  Buser, 
NVEOC. 


2.  SUMMARY  AND  FOLLOW-UP  ACTIONS 


Opening  remarks  were  by  Dr  Rudy  Buser.  In  his  introduction  he  summarized 
the  scientific  and  technology  missions  of  NVEOC  He  indicated  a  strong 
interest  in  exploring  approaches  to  automatic  target  recognition  (ATR), 
particularly  those  which  may  be  useful  for  automatic  recognition  of  military 
targets  located  in  a  dutterea  environment. 

Dr.  Michael  Morris  followed  withan  overview  of  the  work  being  done  at  the 
Institute  of  Optics  on  quantum-limited  imaging  and  image  processing.  He 
discussed  an  approach  to  ATR  in  which  a  quatum-limited  image,  i.e. 
consisting  of  only  a  few  detected  photoevents,  is  correlated  with  a  reference 
function  located  in  computer  memory.  A  key  feature  of  the  method  is  that  it 
provides  a  simple  way  to  reduce  the  amount  of  information  that  must  be 
processed  to  make  reliable  recognition  decisions. 

Next,  three  graduate  students,  working  with  Dr.  Morris  at  the  University  of 
Rochester,  described  the  main  results  obtained  from  their  dissertation 
research.  Thomas  Isberg  discussed  his  work  on  rotation-invariant  filtering 
using  circular-harmonic  expansions,  and  two-staged  template  matching 
Miles  Wernich  presented  work  on  maximum-likihood  image  classification, 
stressing  the  importance  of  filter  designs,  which  are  insentive  to  intrinsic 
object  variations  with  a  given  class.  Edward  Watson  described  methods  to 
upconvert  {both  narrow  and  Broadband)  infrared  images  to  a  visible- 
wavelength  range,  in  which  focal-plane  detector  arrays  are  readily  available. 

Dr.  David  Singer  presented  an  overview  of  the  work  at  NVEOC  on  image 
processing  and  ATR,  with  emphasis  on  model-based,  multi-sensor  algorithm 
development. 

The  final  presentation  was  given  by  Teresa  Kipp,  NVEOC,  on  laser  radar 
imaging.  Effects  of  sampling,  field  of  view,  noise,  ambiguity  interval  on  laser 
imagery  were  illustrated.  Examples  of  the  use  of  range  data  to  for  3-d  surface 
profile  images  were  also  discussed. 

From  the  workshop  the  following  areas  for  continued  interaction  have  been 
identified: 

a.  Specialize  the  University  of  Rochester's  ATR  scheme  for  the  recognition 
of  laser  radar  imagery,  and  compare  its  performance  with  NVEOC 
algorithums.  NVEOC  will  provide  the  University  of  Rochester  with  laser 
radar  imagery. 

b.  Initiate  interaction  with  NVEOC  personnel  on  methods  for  generating 
upconverted  imagery,  and  identify  specific  military  applications. 

c.  Model  the  manner  in  which  speckle  noise  distorts  a  modulated  laser 
radar  beam.  In  turn,  on  the  basis  of  this  work.  NVEOC  will  model  the 
probability  distribution  of  error  in  range  measurements. 
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IMAGE  RECOGNITION  AT  LOW  LIGHT  LEVELS 
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INVARIANT  FILTERING 


Invariant  Image  Recognition 

using 

Quantum-Limited  Images 


Recognition  of  a  Single  Object  within  an 
Input  Scene 

•  Rotation  invariant  Filtering:  Circular-Harmonic 

Filters 

•  Extraction  of  Rotation-,  Position-,  and  Scale- 
Invariant  features:  Radial  Moments  of  Circular- 

Harmonic  Functions 

Recognition  of  Multiple  Objects  within  an 
Input  Scene 

•  Two-Stage  Template  Matching 

•  Two-Stage  Invariant  Filtering  and  Feature 
Extraction 


Future  Applications 

•  Correlation  Tracking 


Invariant  Filtering 


Input  Reference  Correlation 

Function  Output 


g(x',y’) 


C(x,y) 


C(x,y)  remains  unchanged  when  g(x',y') 
is  rotated,  scaled,  or  shifted. 


C(x,y)  attains  its  maximum  value 
when  the  reference  object  is  input 


Rotation-Invariance: 

The  Circular-Harmonic  Expansion 


Expansion 
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Reference  function:  mth  harmonic 
F*  (r,0)  =  F*  (r)  exp(-im0) 


•  Rotation-invariant  correlation 
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Two-Stage  Template  Matching 
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Ref.  Window 

Location:  (x,yj 
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Before  Implementation: 

1.  Specify  required  probabiiites  of  detection 

and  false  alarm. 

2.  Choose  similarity  criterion  and  number  of  photons 

for  each  stage. 

Stage  One 

•  Move  the  location  of  the  ref.  window  to 
each  point  in  the  input  scene. 

•  Sample  using  a  small  number  of  detected 
photons. 

•  Compute  correlation  function  or  invariant 
features. 

•  Apply  threshold  similarity  criterion. 

Stage  Two 

•  Examine  the  locations  that  satisfied  the 
similarity  criterion  using  a  sufficient  number  of 
photons  to  achieve  the  specified  probabilities  of 
detection  and  false  alarm. 


Two-Stage  Template  Matching 
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m 

Ref.  Window 

Location:  (x,y) 

•  ■ 

Determination  of  the  number  of  photons  and 
similarity  criterion  for  each  stage: 

•  The  number  of  photons  in  the  first  stage  should  be 
as  small  as  possible. 

•  The  similarity  criterion  is  chosen  to  achieve  the 
required  probability  of  detection,  while  allowing  as 
few  false  alarms  as  possible. 

•  The  number  of  photons  and  similarity  criterion  in  the 
second  stage  are  chosen  such  that  the  false  alarms 
from  the  first  stage  are  eliminated. 

•  For  a  given  input  scene,  it  is  possible  to  optimize  the 
choice  for  the  similarity  criterion  and  number  of 
photons  in  each  stage. 


Two-Stage  Template  Matching 

Reference  Object:  Tape  Dispenser 


Probability  that  the  similarity  criterion 
will  be  satisfied  at  each  window  offset. 
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IMAGE  CLASSIFICATION 


The  ML  approach 

•  Log-likelihood  ratio 

ln{l(n)}  =  In 

•  Decision  rule 
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d. 


p[  n  I  h  ] 
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Related  strategies 

Minimize  probability  of  error 
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Minimize  cost  of  decision 
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p(H  )  =  a  priori ,  class  j 
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Summary  of  results 

(number  of  detected  photoevents 
to  achieve  1/10000  error  rate) 
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MSLM-based  optical  correlator  configuration 
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Quantum-Limited  Imaging  of 
Upconverted  Infrared  Radiation 


Outline 

Quantum-limited  infrared  detection 
Upconversion  methods 
Narrow  band  imaging 
Broadband  imaging 
Broadband  upconversion 
Upconversion  systems 
Summary 


Infrared  Detection 


Current  detector  technology 

Low  number  of  pixels 
Requires  cooling 

No  mature  methods  for  doing  photon-counting 


Visible  images  by  IR  upconversion 

Allows  use  of  well-developed  visible  detection 
Large  space-bandwidth  product 
Uncooled 

Photon-counting  capability 


Upconversion  Methods 


Incoherent  (Bloembergen,  1959) 


Coherent  (Armstrong 


Pump 

Phase  matching 


Pump  IR 

—  ■■  —  » 


Visible 


No  coherence  requirement 
on  pump. 

Narrow  band  (limited  to 
bandwidth  of  IR  transition) 


et  al,  1962) 


Only  phased-matched  IR 
efficiently  upconverts 

Limited  bandwidth 


Energy  matching 


Narrow  Band  Imaging 


Large  D* 

Active  illumination  is  narrow  band 
Noise  is  broad  band 

Range  extension/increased  resolution 
by  photon-counting 


NEAT 


Broadband  Imaging 


Classical  intensity  levels 

Broadband  imaging  improves 
performance 


Broadband  Upconverei 


ersion 
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Incoherent  Broadband  Upconversion 


Upconversion  Cell 


Dispersive  element  spreads  image  in  upconversion  cell 

Each  wavelength  upconverts  at  different  point  in  cell 

Final  dispersive  lens  system  recombines  visible  light 
into  single  image 


*D.  Faklis  and  G.  M.  Morris,  "Broadband  Imaging  with  Holographic 
Lenses,"  submitted  to  Applied  Optics,  in  review. 


Coherent  Broadband  Upconversion 


Upconversion  Ceil 


Second  lens  produces  plane  waves  in  cell 

Each  wavelength  upconverts  at  different  point, 
producing  a  plane  wave 

Final  dispersive  element  forms  single  image 


G.  M.  Morris  and  D.  A.  Zweig,  "White-Light  Fourier  Transformations," 
In  Optical  Signal  Processing,  (j.  Horner  ed.),  pp  23-71, 

Academic  Press,  New  York  (1987). 


Summary 


Advantages  of  upconversion 

Larger  number  of  pixels  than  direct  IR  imaging 
Uncooled 

Photon-counting  capability 

Active  imaging  uses  narrow  band, 
photon-counting  upconversion 

Passive  imaging  requires  broadband 
upconversion 

Dispersive  optical  elements 
Alkali-metal  vapor  cells 


Monte  Carlo  Image  Compression 
and  Computing  Systems 


Low-Light-Level  Implementation 


Image  Compression  Image  Analysis 


Electronic  Implementation 

Image  Compression  Image  Analysis 


Pattern  Recognition  using 
Quantum-Limited  Images 


Features 

•  Data  Reduction  and  Computation  Speed 

•  Minimal  Computing  Hardware 

•  Intensity-Based  System 


Potential  Applications 

Machine  and  Robot  Vision 
Automatic  Target  Recognition 

Low  light  levels  (passive  night  vision) 
Laser  radar  systems 

Correlation  Tracker  for  Vehicle  Guidance 
Recognition  of  Spectroscopic  Signatures 
Radiological  and  Nuclear  Imaging 
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SUMMARY 


maustnat  Applications,  noc.  im. 

Vol.  960.  Dearborn,  Ml.  June  27-28, 1988 


Pattern  Recognition  using  Photon-Limited  Images 

G.  Michael  Morris,  Thomas  A.  Is  berg,  and  Miles  N.  Wemick 

The  Institute  of  Optics,  University  of  Rochester 
Rochester,  New  York  14627 


j  ABS  f  RACT 

The  spatial  coordinates  of  detected  photoevents  and  the  number  of  detected  photoevems  in  a  given  area  convey  information 
about  the  classical  irradiance  of  the  input  scene.  In  this  paper  the  effectiveness  of  photon-counting  techniques  for  image 
recognition  is  discussed.  A  correlation  signal  is  obtained  by  cross  correlating  a  photon-limited  input  scene  with  a  classical 
intensity  reference  function  stored  in  computer  memory.  Laboratory  experiments  involving  matched  filtering,  rotation-  and 
scale-invariant  image  recognition,  and  image  classification  tre  reported.  For  many  images  it  is  found  that  only  a  sparse 
sampling  of  the  input  is  required  to  obtain  accurate  recognition  decisions,  and  the  digital  processing  of  the  data  is  extremely 
efficient.  Using  available  photon-counting  detection  systems,  the  total  time  required  to  detect,  process,  and  make  a  recognition 
derision  is  typically  on  the  order  of  tens  of  milliseconds.  This  work  has  obvious  applications  in  night  vision,  but  it  is  also 
relevant  to  areas  such  as  process  control,  radiological,  and  nuclear  imaging,  spectroscopy,  robot  vision,  and  vehicle  guidance. 

L  introduction 

f 

Often  input  scenes  contain  a  vast  amount  of  information,  which  tends  to  make  pattern-recognition  riwkiom  laborious  and 
time  consuming.  In  traditional  digital  pattern  recognition  methods,  one  digitizes  the  input  scene  using  a  two-dimensional 
detector,  e.g.  a  solid-state  photodiode  array,  and  a  frame  store.  If  the  detector  consists  of,  say,  a  1000  x  1000  array  of  detection 
elements,  then  one  has  to  process  a  million  points  of  data.  This  is  too  much  information  for  even  very  large  computers  to 
process  in  real  time,  so  one  generally  transforms  the  input  information  into  some  sort  of  feature-space  representation,  e.g. 
through  the  use  of  edge-enhanced  images,  and  makes  the  recognition  decision  based  on  this  reduced  data  set. 

An  alternative  approach  to  the  pattern  recognition  problem  is  to  process  low-light-level  (photon-limited)  input  images 
using  photon-correlation  methods.  In  this  scheme  pbocoevents  are  defected  at  the  maximum  rate  the  detection/computer 
system  can  handle.  One  collects  photoevents  until  there  is  enough  information  about  the  input  scene  to  achieve  an  acceptable  \ 

error  rate  for  the  given  recognition  task.  Cur  studies  indicate  that  in  many  cases  only  a  sparse  sampling  (a  small  number  of  " 

detected  photoevents)  of  the  input  image  is  needed  for  accurate  image  recognition;  hence,  the  time  needed  to  detect,  process,  and  . 
make  a  recognition  decision  can  be  quite  short,  typically  a  few  tens  of  milliseconds.  In  effect  one  is  letting  nature  randomly 
sample  (digitize)  the  input  scene. 

The  problem  that  a  practical  pattern-recognition  system  faces  is  tiiat  it  must  be  highly  sensitive  to  image  features  that  will 
separate  a  given  reference  object  from  the  other  possible  inputs,  yet  at  the  same  time  be  insensitive  to  image  distortions  that 
are  not  important  for  the  recognition  task.  It  is  convenient  to  grw.p  image  distortions  into  two  categories:  geometrical  and 
intrinsic.  Geometrical  distortions  include  variations  such  as  rotation,  scale,  position,  and  aspect  of  an  input  image.  As  an 
example,  one  might  warn  to  recognize  a  given  set  of  images  in  which  the  angular  orientation  and/or  scale  size  can  vary  over  a 
range  of  values.  Intrinsic  variations  are  fundamental  deviations,  not  attributable  to  simple  changes  in  geometry.  In  this  case, 
the  pattern-recognition  task  is  to  identify  the  input  image  as  a  member  of  an  image  class,  rather  than  a  specific  reference 
image. 

In  this  paper,  we  review  tire  research  to  date  on  photon-correlation  systems  in  which  an  input  photon-limited  image  is 
correlated  with  a  fixed  (or  deterministic)  reference  function  stored  in  computer  memory.  The  principal  application  for  these 
correlation  systems  is  automatic  pattern  recognition.  Basically,  one  is  given  a  set  of  possible  reference  images  and  the  goal  is 
to  deride  automatically  (without  human  intervention)  which  of  the  reference  objects  is  most  like  to  input  object.  In  Section  2, 
we  briefly  review  the  statistical  properties  of  photon-limited  imaging  red  photon  correlation.  Section  3  contains  a  brief 
description  of  two-dimensional,  photon-counting  detectors.  In  Section  4  red  5,  correlation  results  obtained  using  re  image  of 
the  reference  function  (matched  filter  case),  circular-harmonic  filters  (rotation  invariance),  Fourier-Mellin  descriptors  (scale  and 
rotation  invariance),  red  maximum -likelihood  filters  (image  classification)  are  summarized. 
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At  km  levels  of  muminatioc,  an  input 
functions,  Le, 


V^(r)  cat  be  represented  as  a  two-dixneasiotul  collection  of  Dirac -delta 


in  which  rk  denotes  the  spatial  coordfaans  of  die  k-tb  detected  photoevent  and  N  is  the  axil  number  of  detected  photoevents. 
Of  oourse,  with  an  arms!  detector  the  photoelectric  comas  are  not  <  points,  bnt  nther  they  occupy  a  finite  area;  the 
effect  can  be  included  by  passing  V^(r)  through  a  linear  system  in  which  the  impulse  response  is  equal  to  the  point  spread 
function  of  the  detector.  Although  one  does  not  want  to  ignore  the  finite  spread  of  the  detected  photoevents,  it  does  lead  to 
helpful  simplification  in  die  notation. 


In  Eq.(l),  the  spatial  coordinates  rk  of  the  k-th  detected  photoevent  are  random  variables.  The  number  of  detected 
photoevents,  N,  may  or  may  not  be  a  random  variable  depending  on  bow  the  experiments  is  performed.  For  example,  N  is  not 
random  far  the  case  when  a  fixed  immber  of  photoevents  is  collected.  On  the  other  hand,  if  photoevents  «e  collected  for  a 
fixed  time  interval  t,  then  from  die  theory  of  photodetection^**  the  conditional  probability  distribution  for  detecting  N 
photoevents  in  the  time  interval  It,  t+t]  from  a  detector  of  area  A,  given  the  classical  image  inadiance  V(r,t),  is  an 
inhomogeneous  Poisson  process  given  by 
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in  which  the  rate  function  X(r,t)  is 
« 

Mr  A)  -  TjV(r,tVhv  ,  (3) 

where  q  denotes  the  quantum  efficiency  of  the  detector,  h  is  Planck's  constant,  and  v  denotes  the  mean  frequency  of  the  inriddnt 
q  uasi  -monochromatic  light  ^ 

In  general,  V(r,t)  is  a  random  process  and  the  conditional  distribution  given  in  Eq.  (2)  must  be  ensemble  averaged  to 
obtain  the  observable  counting  distribution.  For  example,  if  V(r,t)  obeys  negative  exponential  statistics,  as  in  the  case  of 
polarized  quasi-monochro matte  light  emitted  by  a  thermal  source,  then  the  observable  counting  distribution  P(N)  obeys  Bose- 
Einstein  satisfies  when  the  integration  time  is  small  compared  to  the  coherence  rime  of  the  light,  see  e.g..  Ref.  6).  However, 
if  V(r,t)  does  not  fluctuate  significantly,  i Jt„  V(r,t) «  V(r),  as  in  die  case  of  illumination  provided  by  a  well  stabilized  single¬ 
mode  User,  then  counting  distribution  is  simply 
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in  which 
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is  the  avenge  number  of  detected  photoevents.  One  notes  that  the  distribution  in  Eq.  (4)  is  applicable  slso  for  polarized 
thermal  radiation  wheat  is  modi  l«ger  than  the  coherence  time  of  the  light*.  In  this  case  the  fluctuations  in  the  imdiance  are 
put  in  the  time  integration  and  V(r)  iep«gnt»  the  mean  value  at  the  imdiance. 

InEq.  (1)  the  photoevent  coordinates  ate  independent  random  variables.  The  probability  density  function  for 
photoevent  coordinates6  is  directly  proportional  to  the  classical  intensity  V(rk),  Le., 
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22  Correlation  with  «  TVtterministic  Reference  Function 


Our  approach  to  the  pattern-recognition  problem  is  to  calculate  the  correlation  function  formed  by  an  input  photon-limited 
image  aid  a  reference  function  stored  in  computer  memory,  which  describes  the  object  of  interest  Examples  that  illustrate  the 
performance  of  this  method  ere  given  in  Sections  4  and  5.  In  this  section  we  summarize  the  salient  statistical  properties  of  the 
correlation  signal  A  detailed  treatment  of  the  statistical  properties  of  the  correlation  signal  is  given  in  Ref.  7. 

Consider  the  correlation  signal 

C(r)-JdVvV)R(r’+r)  .  '  (7) 


obtained  by  cross  correlating  a  photon-limited  input  scene  V^(r).  given  in  Eq.  (1),  with  a  deterministic  reference  function  R(r) 
stored  in  computer  memory.  Using  Eqs.  (1)  and  (7)  gives 


c<r>-XR(pk+r>  •  {8) 

k-1 

Hence,  the  photon-limited  correlation  signal  C(r)  is  a  random  function  since  the  event  coordinates  are  independent  random 
variables  with  the  probability  density  function  specified  in  Eq.  (6).  The  reference  function  R(r)  may  be  either  real  or  compile. 
In  this  lection  R(r)  is  taken  to  be  a  real-valued  function.  The  results  are  readily  extended  to  the  case  in  which  the  reference 
function  is  complex®.  Also  as  noted  above,  N,  the  number  of  detected  photoevents,  may  or  may  not  be  random  depending  oA 
how  the  input  image  is  sampled.  Here,  we  take  the  number  of  detected  photoevents  to  be  fixed. 

To  calculate  C(r),  one  uses  the  spatial  coordinates  of  a  given  detected  photoevent  as  an  address.  The  offset  coordinated  r 
defines  the  location  of  the  reference-function  window  within  the  input  scene.  The  procedure  to  calculate  C(r)  is  to  look  up  the 
value  of  the  reference  function  stored  at  the  address  specified  by  r  +  rk,  and  place  that  value  into  an  accumulator;  this 
operation  is  repeated  for  all  N  detected  photoevents.  The  recognition  decision  is  based  on  the  resulting  value  of  C(r). 

In  a  typical  pattern-recognition  problem,  one  would  generally  expect  that  the  number  of  detected  photoevents  needed  to 
make  an  accurate  recognition  decision  would  be  rather  large,  at  least  a  few  hundred  to  a  few  thousand  photoevents.  As  N  gets 
large,  the  correlation  signal  in  Eq.  (8)  tends  to  a  Gaussian  process.  In  the  limit  of  large  N,  the  probability  density  function 
for  C(r)  is  given  by 


(C(r)  -  <C(r)>]1 
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in  which  the  expected  value  of  the  correlation  signal  <C(r)>  is 


<C(r)> -NjdV ptrlVfr*)]  R(r'  ♦  r)  .  M) 

the  variance  is 

c*  -  N  J  dV  p[r*JV(r*)]  rV  +  r)  -  .  01) 

end  pfrTVfr*)]  is  given  in  Eq.  (6).  Notice  thii  <C(r)>  h  directly  proportional  to  the  correlation  function  between  the  high- 
bght-lerd  (or  classical  intensity)  input  scene  and  reference  function  R(r). 

?  ?  i  Probability  nf  Deteetinw  «nd  Fake  Alarm  The  detection  problem  can  be  formulated  by  using  the  statistical  theory 

of  hypotheses  testing9.  On  the  basis  of  the  cotrelatian  signal  C(r),  one  must  choose  between  two  hypotheses:  the  null 
hypothesis,  Hq- the  reference  function  R(r)  is  hoi  present  in  the  input  scene  V(r);  or  the  positive  hypothesis,  H  j  -  the 
reference  function  is  contained  in  V(r). 

Under  hypothesis  Hq.  the  probability  density  function  of  the  correlation  signal  is  denoted  by  Pq(C)  - 
PtCfrJtVfr')  -  N(r*)l.  where  N(r*)  represents  a  noise  (or  false)  image.  Under  hypothesis  Hj,  the  probability  density  function  of 
the  correlation  signal  is  denoted  by  Pj(C)  -  PIC(r)|V(r')  -  R(r*>],  where  R(r*)  is  the  reference  image. 

Operationally,  the  observer  usually  sets  a  threshold  value  Cy  for  the  correlation  signal.  If  C(r)  >  Cy,  hypothesis  H]  is 
chosen.  Similarly  if  C(r)  <  Cj.  the  observer  chooses  hypothesis  Hq.  However,  because  of  the  statistical  nature  of  the  signal, 
the  observer  occasionally  mates  an  error,  regardless  of  the  value  of  Cy.  The  probability  of  choosing  Hj  when  Hq  is  true  is 
called  the  probability  of  false  alarm  and  is  given  by 


Pft-JdCP0(C)  .  (12) 

Cr 

The  probability  of  choosing  H  j  when  H  j  is  true  is  called  the  probability  of  detection. 

P4-jdCP,(C).  (13)  j 

Cr 

3.  TWO-DIMENSIONAL.  PHOTON-COUNTING  DETECTORS 

Almost  all  the  two-dimensional  photon-counting  detectors  that  have  been  developed  use  a  mioochannel  image 
intensifier1®*1 1  in  cur-wi*  with  some  type  of  anode  assembly  to  record  the  position  coordinates  of  the  event  A  number  of 
node  assemblies  have  been  used  including  sQicon-intensified-target  television  cameras12*14,  self-scanned  (CCD)  detector 
arrays 15-20  crossed- wire-grid  anodes21,  multi-anode  anays22,  resistive  anodes2^*22  wedge-and-strip  anodes2®*^1,  and  gray- 
coded  masks  used  with  a  band  of  photomultipliers  ^2. 

A  schematic  of  a  resistive-anode-type  detector  is  shown  in  Fig.  I.  An  incident  photon  ejects  an  electron  from  a 
photocathode.  The  ejected  photoelectron  is  directed  into  a  stack  of  mioochannel  plates  (typically  ananged  in  a  V.  and/or  2- 
stack  to  prevent  ion  feedback)  to  achieve  »  electron  gain  C  of  approximately  106  to  10°.  The  resulting  charge  pulse  is 
collected  by  the  resistive-anode  assembly.  The  resistive  layer  is  terminated  by  electrodes  at  three  cr  four  locations  around  its 
perimeter,  when  provide  the  signals  for  the  centroiding  algorithm.  When  coupled  to  position-computing  electronics,  the 
detection  system  is  generally  capable  of  operating  at  count  rates  up  to  approximately  100,000  counts  per  second  with  a  spatial 
resolution  of  approximately  400  x  400 elements24. 
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Fig.  1.  Schematic  diagram  of  a  resistive-anode,  photon-counting  detection  system,. 


The  choice  of  the  anode  assembly,  of  course,  depends  on  the  requirements  imposed  by  the  application.  For  example, 
detection  systems  that  operate  with  only  one  detected  photoevent  in  the  device  at  a  time  can  be  used  to  collect  a  specific 
number  of  photoevents;  hence,  N  in  Eq.  (1)  can  be  fixed,  instead  of  random.  Theses  detectors  can  provide  both  the  spatial 
coordinates  and  time-of-arriva!  information  of  the  detected  photoevents.  On  the  other  hand,  detection  systems  that  utilize  a 
detector  arrays  or  a  television  tube  to  read  out  the  position  coordinates  of  detected  photoevents  are  generally  preferable  in  high 
speed  applications  that  involve  moving  objects  or  pulsed  light  sources.  In  these  systems  many  events  can  be  detected  in  a 
short  time  interval  and  then  read  out  in  a  raster  format,  die  time -of  -arrival  information  of  detected  photoevents  is  not  available, 
and  the  number  of  detected  photoevents  for  a  fixed  time  interval  x  is  a  random  variable.  "  . 


The  results  in  Section  2  are  applicable  far  general  real-valued  reference  functions  and  input  scenes^.  The  capabilities  of 
this  low-light-level  approach  to  pattern  recognition  can  be  predicted  theoretically  by  studying  the  probability  density  functions 
of  the  correlation  signal  when  the  input  scene  (1)  matches  the  reference  function  stored  in  computer  memory,  and  (2)  is  a 
typical  background  image  encountered  in  the  given  application.  From  these  probability  density  function,  one  can  determine 
die  number  of  detected  photoevents  that  is  required  to  achieve  a  given  probability  of  detection  and  false  alarm. 

In  this  Section,  we  consider  the  correlation  signal  obtained  when  the  input  scene  is  correlated  with  a  classical  intensity 
image  of  the  reference  object  In  the  case  the  correlation  output  C(r)  in  Eq.  (8),  corresponds  to  that  of  a  matched  filter.  When 
N,  the  number  of  detected  photoevents,  is  a  fixed  number,  the  mean  value  and  variance  of  the  correlation  signal  are  given  by 
Eqs.  (10)  and  (11),  respectively. 

As  an  example,  let  us  consider  the  recognition  of  detailed  images,  such  as  the  engraved  portraits  of  George  Washington, 
Abraham  Lincoln,  and  Andrew  Jackson  shown  in  Fig.  2.  The  photographs  in  Fig.  2  were  obtained  by  imaging  portraits  from 
U.  S.  currency  onto  a  two-dimensional,  photon-couiuing  detector  [Electro-Optical  Products  Div„  ITT  Corporation,  Model 
F4146M],  The  detector  was  connected  to  position-computing  electronics  [Surface  Science  Laboratories,  Model  2401]  to 
determine  the  spatial  coordinates  of  detected  photoevents.  The  spatial  coordinates  of  the  detected  events  were  digitized  to  a 
spatial  resolution  of  256  x  256  pixels  and  then  sent  to  a  microcomputer  system  for  display.  Illumination  was  provided  by 
fluorescent  room  lights.  Neutral  density  filters  were  inserted  between  the  imaging  lens  and  the  detector  to  reduce  the  count  rate 
to  approxinutely  50,000  counts  per  second.  In  each  image  a  fixed  number  of  detected  photoevents  was  collected.  The  number 
of  detected  events  fcr  the  images  in  a  given  row  are  indicated  along  the  left-hand  side. 
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Fig.  2.  Images  of  engraved  portraits  obtained  using  a  two-dimensional,  photon-counting  detection  system; 
first  column,  portrait  of  George  Washington;  second  column,  Abraham  Lincoln;  third  column,  Andrew 
Jackson.  N  is  die  number  of  detfcred  pbotoevents  over  the  entire  image:  top  row,  N  -  20  million;  middle 
row,  N  -  4,000;  bottom  row,  N  -  1,000.  The  spatial  coordinates  of  each  detected  photoevent  are  digitized  to 
8-bit  accuracy. 

Hie  probability  density  functions  of  the  correlation  signal  in  Eq.  (8)  are  calculated  using  the  images  in  the  top  row  of  Fig. 
2.  For  detailed  imagery  one  expects  that  the  number  of  detected  photoevents  will  need  to  be  relatively  large  for  reliable 
recognition;  hence,  the  approximation  that  the  probability  density  function  of  the  correlation  is  Gaussian  distributed  should  be 
fairly  accurate. 

Since  the  reference  and  input  images  have  the  same  area  A,  we  take  the  reference-window  offset  r- 0.  The  mean  value  and 
variance  of  the  Gaussian -distributed  correlation  signal  is  computed  for  each  mjxit  object  using  Eqs.  (10)  and  (11),  respectively. 
By  using  the  mean  values  and  variances  for  the  different  input  objects,  one  can  calculate  the  probability  of  detection  and 
probability  of  false  alarm  versus  N,.the  number  of  detected  photoevents.  Figure  3  contains  ROC  curves  for  the  portraits  of  for  ‘ 
different  values  of  N.  The  portrait  of  Washington  was  used  as  the  reference  function.  From  the  ROC  curves,  one  sees  that 
excellent  discrimination  can  be  obtained  by  using  less  than  one  thousand  detected  photoevents  (see  bottom  tow  of  Fig.  2).  It  is 
this  extreme  data  reduction  that  enables  one  to  achieve  real-time  operation. 
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Fig.  3.  ROC  curves  for  the  portraits  of  Washington  and  Lincoln  for  different  values  of  N. 


In  operation,  the  recognition  decision  is  based  on  a  single  realization  of  the  correlation  signal.  N  photoevents  from  the 
input  image  are  detected,  and  the  resulting  correlation  signal  is  compared  with  a  threshold  value  Cj.  If  the  value  of  the 
correlation  signal  exceeds  Cj,  the  reference  image  is  said  to  be  present  in  the  input;  if  not,  the  reference  object  is  said  to  be 
absent.  Excellent  agreement  between  theoretical  predictions  and  laboratory  experiments  have  been  observed.  Note  that  at  a  rate 
of  50,000  counts/ sec,  the  total  time  to  detect,  process,  and  make  a  recognition  decision  using  1000  detected  photoevents  is 
approximately  20  milliseconds.  ,1 

I  - 

It  is  important  to  note  that  in  the  theoretical  predictions  for  the  correlation  signal,  no  corrections  were  make  for  additive 
noise  or  dead-time  effects;  these  effects  are  simply  not  important  when  the  count  rate  is  50.000  counts/sec  and  N  is  a  a  few 
thousand  counts  or  less.  In  the  experiments  the  dark -count  rate  is  approximately  50  counts/sec  at  room  temperature  (a  bi-alkali 
photocathode  is  used  in  the  detector).  At  a  rate  of  50,000  counts/sec,  on  average,  there  is  only  one  detected  photoevent  out  of  a 
thousand  that  is  associated  with  additive  noise;  hence,  the  contribution  due  to  additive  noise  is  negligible. 

5.  DISTORTION-INVARIANT  FILTERS 

In  most  applications  it  is  desirable  to  have  an  automatic  recognition  system  that  can  tolerate  certain  variations  in  input 
images,  which  are  not  important  as  far  as  a  recognition  decision  is  concerned.  These  variations  can  be  divided  into  two  basic 
categories:  geometrical  distortions  of  the  image  (e.g.,  rotation,  scale,  and  position)  and  intrinsic  variations  of  the  image  (e.g. 
changes  in  illumination,  image  clutter  and  object  occlusions) .  One  could  approach  this  problem  by  using  multiplexed  filters 
in  which  a  separate  reference  function  (filter)  is  used  for  each  scale  and  orientation  of  the  input  object,  but  this  leads  to  a 
computational-intensive  system  design  that  is  difficult  to  implement 

A  more  elegant  approach  to  the  problem  of  image  distortion  is  to  choose  a  reference  function  R(r)  so  that  the  correlation, 
output  is  invariant  to  the  distortion  of  the  input  image.  Mellin  transforms,  which  have  scale-  and  position-invariant  properties 
have  been  used3^.  Circular-harmonic  filters,  initially  suggested  by  Hsu  et  <a/.3^’  have  proved  useful  for  rotation-invariant  - 
pattern  recognition  8,36-38  Fourier-Mellin  descriptors  for  rotation,  scale  and  position  invariance  have  also  been  reported39' 

42,  Other  approaches  to  distortion-invariant  filtering  include:  synthetic  discriminant  filters^3,  and  the  "lock  and  tumbler" 

filter^. 


tatriusic  variations  within  images  can  be  treated  as  a  problem  m  image  deification,  ^tbctepm juMfieuto 

trfwme  cSbwy  or  class  of  images.  A  broader  interpretation  of  image  classificatjaj  revefr*^ 
famte  recognition  in  realistic  environments  is  always,  in  fact,  a  sorting  problem.  Generally,  input  *£*“  J 
cta£^3£»s  fat  flluraination,  viewing  angle,  defects,  occlusions,  etc.  By  treating  a  set  of  images  of  a  P£»  object  *  * 
fanage'dass,  one  can  formulate  the  problem  of  image  recognition  in  realistic  environments  as  an  image  dnstficanon  problem. 

«<v  most  rwHablf  statute  nJ  <4»««fie»rinn  algorithms  m  those  proposed  by  Fntamaga  and  Koona*  .Foley and 
g«Mwnn46  and  the  Hotelling  trace  algorithm*7,  all  of  which  have  been  used  in  imagc-clasrificatina  schemes4  ' 

In  *j«  section  we  consider  three  examples  of  distortion-invariant  filtering  it  low  light  levels.  Experimental  results  for 
circular-harmonic  filters,  Fomier-Mellm  descriptors,  and  muimum-libelihood  image  classification  ait  given. 


t  Rotation-Invariance  using  fTtrcular-Harmonic  Filters 

Any  two-dimensional  function  f(r,0)  can  be  represented  in  terms  of  fas  circular-harmonic  components  as  follows: 


f(r,0)  -  £  F.(r)  t™*  ,  (14) 

where 

2  Jt 

FJr)--J-ff(r.0)e*in,e  d0  .  '  (15) 

2*o 

In  Eq.  (14),  Fm(r,0)  -  Fm(r)exp(im0)  is  said  to  be  the  m-th  circular-harmonic  component  of  the  functioo  f(r,0).  Rotation- 
invariant  filtering  can  be  achieved  by  taking  the  reference  function  R(r),  in  Eq.  (8),  to  be  the  complex  conjugate  of  a  single  (or 
multiple)  circular-harmonic  components)  of  the  reference  object,  e.g., 

*(r)-F;(r,0)-F;(r)e-im0  .  (16) 

When  the  offset  coordinate  r  -  0,  the  correlation  signal,  C(r,a),  is  an  inner  product  of  an  input  function  g(r,0+a)  rotated  by  an 
angle  a  with  respect  to  the  reference  function  Fm*(r,0):  * 

2rt«o  I 

C(0,a)  -  JJ  g(r,0+a)  F;  (r,0)  rdidG  .  (17) 

00 

Using  the  expansion  in  Eq.  (14)  for  g(r,0+a)  and  substituting  into  Eq.  (17)  yields 


C(0,a)  -  2rt  eima  J  GB(r)  Fi  (r)  rdr  , 
0 


(18) 


in  which  the  modulus  of  C(0,a)  attains  its  maximum  value  when  Gm(r)  -  Fm(r),  as  given  by  the  Schwartz  inequality.  Notice 
that  only  the  m-th  circular-harmonic  component  of  the  input  contributes  to  C(0,a)  in  Eq.  (17)  owing  to  the  orthogonality  of 
the  0- integration,  and  that  the  rotation  angle  a  appears  only  in  the  phase  tom  expfima).  Hence,  the  modulus  of  C(0,a)  is 
independent  of  the  rotation  angle  of  the  input  image.  If  the  reference  image  is  input,  its  orientation  can  be  obtained  from  the 
ratio  of  the  real  and  imaginary  pans  of  the  correlation  signal.  The  rotation  angle  a  is  given  by 


q.J— aH-lfiSi . 

»  LRe{C(0,a)}J 


(19) 


which  can  be  calculated  easily  when  the  correlation  is  implemented  digitally.  The  rotation  angle  a  is  far  more  difficult  to 
obtain  in  *n  optic*]  implementation  because  molt  detection  scheme*  are  intensity  based.  . 

Thr  magnitude  of  correlation  peak  will  depend  on  the  particular  harmonic  that  is  chosen i  as  the  reference  functKjn.  The 
payntnde  of  C(r,a)  also  depends  on  the  location  of  the  pant  about  which  the  reference  object  is  expanded;  tins  location  is 
referred  to  as  the  expansion  center.  The  magnitude  of  C(i‘Xt)  will  be  an  absolute  maximum  only  when  the  'proper  center  is 
dmen  as  die  expansion  center  Several  investigations  using  the  centroid  of  die  image  (rather  than  the  proper  center)  as  an 
rTp»««inn  center  have  been  denmstrated  to  be  effective  for  rotation-invariant  image  recognition^^7. 


Experiments  to  test  the  recognition  capabilities  of  the  photon-limited  correlation  scheme  when  a  complex  cncular- 
hannonic  filter  is  used  as  the  function  R(r)  were  performed.  In  the  experiments  35-mm-format  input  scenes, 

illuminated  by  an  incoherent  light  source,  were  imaged  onto  an  ITT  Model  F4146M  photon-counting  detector.  Neutral  density 
filters  were  inserted  to  obtain  a  count  rate  of  approximately  30,000  counts/sec.  The  (x,y)-spatial  coordinates  of  the  detected 
phosoevents  were  digitized  to  8-bit  accuracy  and  sent  to  a  digital  computer  for  processing.  The  number  of  detected 
photoevents,  N,  (see  Eq.  (8))  was  fixed. 


Figure  4  shows  die  input  images,  the  reference  function,  and  results  from  the  correlation  experiments.  The  second  (m»2) 
circular-harmonic  of  the  vise  grips,  computed  about  the  centroid  of  the  object,  is  taken  as  the  reference  function.  The  pliers  and 
the  movable-jaw  wrench  are  used  as  false  objects. 


Fig.  4.  Rotation-invariant  filtering  using  a  circular-harmonic  filter,  (a)  Images  used  in  correlation 
experiments,  (b)  the  second-circular  harmonic  of  the  vise  grips  is  used  as  the  reference  function;  (c) 
correlation  results  obtained  using  3000  detected  photoevents. 


To  lest  the  performance  of  the  drcular-harmooic  filter  at  law  light  levels,  conditions  between  photon-limited  tool 
images  end  the  complex  circular-harmonic  reference  function  stored  in  computer  memory  were  computed.  Measurements  wet 
taken  with  the  input  reference  object  at  varioos  orientations  (0, 90, 180  and  270  degrees  with  respect  to  the  reference);  the  fal 
objects  were  imaged  onto  the  detector  at  the  orientations  shown  in  Fig.  4.  Histograms  of  the  correlation  v aloes  obtained  usi 
3000  delected  photoevena  are  shown  in  Fig.  4.  The  solid  carves  are  the  theoretical  predictions®  of  the  probability  density 
fmetionfor  the  squared  modulus  of  the  correlation  signal.  IQ2  -  |C(0,a)|2.  For  each  input  object,  one  thousand  realizations 
of  the  U¥irlition  signal  were  used  to  calculate  the  histogram.  In  accord  with  theory,  the  magnitude  of  the  correlation  signal 
was  found  to  be  independent  of  the  object-rotation  angle  cl 

In  Fig.  5,  ROC  curves  are  plotted  for  the  vise  grips  (reference  object)  and  the  movible-jaw  wrench  (false  object).  Note 
that  with  N  -  3000  delected  photoevents,  the  probability  of  error  is  extremely  small  (approximately  one  error  in  100,000 
realizations).  Based  on  theoretical  predictions,  one  finds  that  if  the  image  of  the  vise  grips  were  used  as  the  reference  fane  dot 
(*-*-.  matched-filter  case),  the  same  discrimination  capability  can  be  achieved  with  approximately  1000  detected 
photoevents.  Hence,  by  increasing  the  number  of  detected  events  by  only  a  factor  of  three,  one  can  add  rotational  invariance 
into  the  recognition  system. 


Itaatian-,  scale-  and  position-invariant  image  recognition  can  be  accomplished  through  the  use  of  Fourier-Mellin 
“Scnpton  (FMD's).  In  this  method  either  the  radial  moments  or  the  radial  Mellin  transform  of  dominant  terms  in  the  circular 
harmonic  expmsion  of  a  reference  object  are  computed  about  the  centroid  of  the  image.  The  modulus  of  these  descriptors  is 
used  to  define  invariant  features.  The  invariant  features  of  an  input  scene  are  compared  with  the  corresponding  features  of  the 
reference  object  A  recognition  decision  is  based  on  the  sum  of  the  squared  differences  of  the  features,  D2.  If  D2  is  less  than 


aome  pre-determined  threshold  value,  the  input  is  said  to  be  the  same  as  the  reference.  If  D2  the  threshold,  then  the 

input  is  said  to  be  different  than  the  reference. 


The  invariant  features  $Jin  of  the  input  image  V(r)  -  V(r,9)  are  given  by 


where  the  Fouricr-Mcllin  descriptor!  Mt  tw  are 


2k  — 

M*.  -  J  J  V(r,8)  I**1  e4™6  drd0  .  (21) 

00 

The  modulus  of  Mfm  is  invariant  with  respect  to  rotation  end  charges  in  position  of  the  input  image.  If  the  MeUin 
transform  variable  s  is  pure  imaginary,  the  modulus  of  ^  is  also  scale  invariant.  On  the  other  band,  if  the  transform 
variable  s  is  real  (&g.  an  integer),  the  normalization  by  the  0-th  order  descriptor  [Eq .  (20)]  provides  scale  invariance. 

Invariant  features  can  be  estimated  using  quantum- limited  images  in  the  following  manner.  If  one  chooses  the  reference 
function,  which  is  stored  in  computer  memory,  to  be 

R(r,0)  - 1*'2  e4®9  ,  (22) 

where  the  coordinates  (r,0)  are  taken  with  respect  to  the  centroid  of  the  input  image,  the  mean  value  of  the  subsequent  photon- 
limited  correlation  signal  is  given  by 


<C-> 


2rt  — 

N  J  J  Vfr.ejr^e4®6  drd9 
00 _ 


2ft  n 

J  J  V(r,8)idn» 
00 


where  N  is  the  number  of  detected  photoevents.  Notice  that  <Cy  in>  is  directly  proportional  to  the  classical  intensity 
descriptor  Ms>m  in  Eq.  (21).  Also, 


l<CWB>l  MuJ  . 


(24) 


\ 

t> 


if  different  photoevents  are  used  to  compute  <C(s)>  and  <C(0)>. 


I 


The  recognition  capabilities  of  the  photon-limited  estimates  of  the  FMD's  were  tested  in  laboratory  experiments.  A  set  of 
transparencies  (35-mm  format)  were  made  using  engraved  portraits  from  U.  S.  currency,  at  relative  magnifications  of  1.0, 1.25, 
1.5  and  1.75.  These  transparencies  (input  at  various  orientations)  were  imaged  onto  the  twodimensional,  photon-counting 
detector.  Five  thousand  detected  photoevents  were  used  to  determine  the  location  of  the  centroid  of  the  input  object  in  each 
case.  Hie  invariant  features  were  computed  using  the  reference  function  in  Eq.  (57)  with  s-2.  The  distance  in  feature  space 
was  computed  between  the  quantum- limited  input  image  and  the  reference  image  (George  Washington  with  a  relative 
magnification  of  unity).  To  test  the  theoretical  predictions,  one  thousand  measurements  of  the  feature-space  distance  D^, 


m 


input  .ref.  ,2 
m  *  *2,nr 


(25) 


were  performed  for  each  input  image  to  provide  accurate  estimates  for  the  mean  values  and  standard  deviations  of  D2. 

It  was  found  that  radial  moments  of  three  circular  harmonics  (m  •  1, 2  and  6)  were  sufficient  to  provide  an  error  rate  of  1 
x  10*5  when  5000  detected  photoevents  were  used  to  construct  the  estimates  of  the  FMD  for  each  value  of  m4^.  The 
probability  density  functions  for  l£,  based  on  experimental  values  for  the  mean  values  and  variances  of  for  the  different 
input  images,  are  shown  in  Fig.  6.  In  Fig.  6,  a  threshold  value  -  0.08  provides  the  error  rate  of  1x10*3  stated  above. 
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Fig.  6.  Probability  density  functions  for  D2,  based  on  experimental  measurements  of  the  mean  value  and 
variance  of  D2  for  the  images  of  Washington,  Lincoln ,  and  Jackson.  The  invariant  features  <(>2,m  (m  -  1, 2, 
and  6)  of  Washington  were  used  as  the  reference. 

1 

ti 

53..  Image  Classification  , 

In  certain  applications  it  may  be  desirable  to  ascertain  the  category  or  class  of  images  to  which  an  input  image  belongs. 
An  image  class  may,  for  example,  include  images  of  a  single  scene  in  which  variations  in  background  content,  illumination 
angle,  arid  other  attributes  are  included.  Printed  characters  in  various  fonts  may  likewise  compose  a  class  of  images.  In  this 
lection  photon-limited  image  correlation  is  applied  to  the  problem  of  sorting  two  image  classes.  The  high  rate  at  which 
images  can  be'idendfied  arises  from  the  efficiency  of  the  photon-correlation  calculation. 

The  photon-limited  image  consists  of  a  histogram  of  the  spatial  coordinates  of  detected  photoevents.  Hence,  a  digitized 
photon-limited  image  takes  the  form  of  a  two-dimensional  array  n.  At  zero  offset,  the  cross  correlation  between  of  a  photon- 
limited  input  image  n  and  a  reference  function  R(r)  is  given  by 

c  “  S  "i  R(|V  *  P6> 


where  r  j  represents  the  coordinates  of  the  i-th  pixel  of  the  reference  image  and  nj  is  the  number  of  detected  photoevents  in  that 
pixel  In  practice,  most  of  the  nj’s  are  zero;  hence,  the  calculation  consists  simply  of  a  summation  of  the  values  of  the 
reference  function  R(r)  sampled  at  the  spatial  coordinates  of  the  detected  photoevents. 

Our  goal  is  to  construct  a  reference  function  R(r)  that,  when  applied  according  to  Eq.  (26),  yields  useful  information  for 
class  discrimination.  The  maximum-likelihood  approach  is  found  to  be  particularly  well  suited  to  the  problem  of  sorting  two 
classes  of  photon-limited  images^1. 


The  maximum-likelihood  decision  criterion*^  is  to  choose  the  hypothesis  comsponding  to  the  state  that  was  most  likely 
to  hare  given  rise  to  the  observed  data.  The  central  quantity  in  the  theory  is  the  likelihood  ratio  l,  given  by 


Mb)- 


P{i»1H(1)] 
pMH®} 


(27) 


when  ■  represents  the  photon- limited  input  image  «1  hypothesis  H®  Q-12)  indicates  that  the  input  image  is  a  member  of 
class  j.  The  procedure  that  we  will  use  to  classify  an  input  image  is  as  follows.  When  a  photon-limited  image  n  is  input,  the 
ratio  Mp)  is  calculated.  To  select  the  most  probable  source  of  the  detected  photoeveno,  class  1  is  chosen  when  Mb)  >1  and 
class  2  is  chosen  when  /(■)<!. 


Frequently,  for  convenience,  the  log-likelihood  ratio  ta{f(n)}  replaces  the  likelihood  ratio  Mp)  as  the  decision  parameter. 
Wemick  and  Morris*  *  have  shown  that  bi{i(a)}  is  given  approximately  by 


h{/(n)}  s  £  n,  fat  (r-Y)  +  £  «“2]i '  [ml  V  * 
i  1  2Ji  i 


(28) 


where 


(29) 


denotes  the  nradiance  of  the  »-th  pixel  of  the  k-th  training  image  in  class  j,  Mj  is  the  number  of  training  images  in  class 
j,  and  [mj]j  is  the  irradiance  in  the  i-th  pixel  of  the  average  image  from  class  j. 

Comparing  Eqs.  (26)  and  (28),  it  is  seen  that  the  natural  logarithm  term  serves  as  the  reference  function  for  maximum- 
likelihood  image  classification  at  low  light  levels.  The  term  involving  the  difference  of  class  means  is  a  bias,  which  is 
independent  of  the  input  image;  it  simply  changes  the  threshold  value  of  in{J(n)}  that  is  used  to  decide  between  the  two  * 
classes.  Therefore,  in  operation,  evaluation  of  the  log-likelihood  ratio  reduces  to  the  table-lookup- and- addition  procedure  that 
we  seek.  I 

Experiments  to  compare  the  performance  of  various  image-classification  methods  at  low  light  levels  have  been 
performed**  using  images  of  characters  and  tools.  Representative  members  of  the  training  sets  used  in  the  character- 
recognition  experiments  are  shown  in  Fig.  7.  Reference  functions  were  constructed  for  maximum-likelihood  classification,  the 
Fukunaga-Koontz  transform,  and  average  filtering.  The  reference  function  far  maximum-likelihood  image  ri«tifirarirw  is 
shown  in  Fig.  8. 

To  evaluate  the  performance  of  the  various  algorithms  we  use  as  a  figure  of  merit  N  -  the  number  of  detected  photoevents 
required  to  reduce  the  probability  of  error  (Le,  the  fraction  of  decisions  that  are  incorrect)  to  1(H.  The  results  obtained  for  the 
various  classification  methods  are  as  follows:  maximum  likelihood,  N  -  146;  difference  of  means,  N  -  246;  Fukunaga- 
Koontz,  N  -  965.  Note  that  the  maximum-likelihood  solution,  Eq.  (28),  yields  the  best  results. 

The  classification  techniques  illustrated  above  are  designed  for  sorting  of  two  image  classes.  The  gt-n^iiraHnn  to 
multiple  desses  is  most  readily  obtained  through  application  of  a  pairwise-voting  logic,  in  which  pain  of  image  classes  are 
sucMssively  compared.  On  each  pairwise  decision,  one  class  is  eiimmited,  Thus,  if  K  classes  are  to  considered,  then  (K-l) 
correlations  must  be  performed,  hi  a  hard-wired  system,  many  reference  function  may  be  applied  in  parallel  to  the  incoming 
photoevent  data.  The  resulting  correlation  values  may  be  passed  to  a  system  that  implements  the  voting  logic. 
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Fig.  8,  Reference  function  for  naxinuun- likelihood  classification  of  Fi  and  R'*  Tk*  f.m  ^ . 

positive  and  negative  values;  a  bias  has  been  added  faSkypu^  funcnon  Contams  b0th 


P  SUMMARY 
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The  «t»wttiril  properties  of  the  correlation  signil  ire  summarized  in  Section  2.  The  system  to  compute  the  correlation 
signal  C(r),  fanned  by  a  photon-limited  inpot  scene  and  a  deterministic  reference  function,  is  simply  an  optical 
implementation  of  a  Monte  Carlo  scheme  to  calculate  definite  integrals. 


Several  reference  functions  for  pattern  recognition  are  examined  Examples  of  seem  matching  when  the  input  image  is 
with  an  image  of  the  reference  object  are  given  in  Section  4  (see  Figs.  2  and  3).  In  Section  5,  distoroon-mvanant 


fliers  ate  considered  Experitmntal  resulB  that  illustrate  the  use  of  circular-harmonic  filters  for  rotation  invariance  (Figs.  4 
and  5),  relation,  fc«V  and  position  invariance  (Fig.  6)  and  image  classification  (Figs.  7, 8  and  Table  1)  are  given. 


Tbe  hardware  requirements  to  implement  this  photon -correlation  scheme  are  minimal.  The  detection  system  (see  Section 
3)  consists  of  a  two-dimensional,  photon-cocnting  detector  and  position-computing  electronics,  which  is  connected  to  a  ' 
microcomputer.  Tbe  reference  function  is  stored  in  computer  memory.  Since  the  photon-correlation  systemis  intensity-breed, 
it  ere  operate  with  either  spatially  coherent  or  noncoherent  radiation.  \ 


There  me  a  number  of  potential  applications  for  the  methods  discussed  in  this  paper.  These  include:  machine  and  robot 
vision,  target  recognition  from  video  monitors,  correlation  tracking  for  vehicle  guidance,  active  or  passive  night  vision, 
automatic  recognition  of  spectral  signatures,  radiological  imaging,  and  range  extension  for  laser  radars. 
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